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Abstract: We present a flow-control technique in traffic-light intersections, aiming at regulating queue 
lengths to given reference setpoints. The technique is based on multivariable integrators with adaptive 
gains, computed at each control cycle by assessing the IPA gradients of the plant functions. Moreover, 
the IPA gradients are computable on-line despite the absence of detailed models of the traffic flows. 
The technique is applied to a two-intersection system where it exhibits robustness with respect to 
modeling uncertainties and computing errors, thereby permitting us to simplify the on-line computations 
perhaps at the expense of accuracy while achieving the desired tracking. We compare, by simulation, the 
performance of a centralized, joint two-intersection control with distributed control of each intersection 
separately, and show similar performance of the two control schemes for a range of parameters. 
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1. INTRODUCTION 

Infinitesimal Perturbation Analysis (IPA) has been established 
as a sample-based technique for sensitivity analysis of Dis¬ 
crete Event Dynamic Systems (DEDS). Specifically, it gives 
formulas or algorithms for the sample derivatives (gradients) 
of performance functions with respect to structural and control 
variables. One of its salient features is the simplicity and com¬ 
putational efficiency of its algorithms in a class of DEDS which 
follow formal rules for propagation of perturbations in a net¬ 
work. Furthermore, the algorithms are based on the monitoring 
and observation of sample paths associated with the evolving 
state of the system, and whenever these are measurable in real 
time, the IPA has a potential in control. For extensive presen¬ 
tations of the IPA technique, please see Ho and Cao (1991); 
Glasserman (1991); Cassandras and Lafortune (1999). 

The principal application-domain of IPA has been in queueing 
networks. However, in recent years there has been a growing 
interest in fluid queues and their generalization to a class of 
Stochastic Hybrid Systems (SHS). There are three reasons for 
that: (i) the algorithms for IPA often are simpler in the SHS 
setting than in their DEDS equivalent models; (ii) they often 
require only an observation of the system’s sample paths but 
not detailed or explicit knowledge of the underlying probability 
law and hence may be implementable on-line; and (iii) the 
IPA derivative estimators are unbiased in a larger class of 
problems in the SHS framework as compared to the setting 
of DEDS. Initial developments of IPA in the SHS framework 
were presented in Cassandras et al. (2002), and more-recent 
general results as well as surveys can be found in Cassandras 
et al. (2010); Yao and Cassandras (2011). 
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We point out that representing a discrete event system by an 
SHS model may introduce errors in the performance evaluation, 
and this point was addressed in the aforementioned papers and 
references therein via simulation-based parameter optimiza¬ 
tion. In most of these experiments the underlying system that 
provided the sample paths was a DEDS, but the algorithms 
for the IPA had been derived from the SHS model. Despite 
this discrepancy the optimization techniques converged to min¬ 
imum points (or local minima), thereby suggesting a degree of 
robustness of them with respect to errors in the computed IPA 
gradients. This point plays a role in the later developments in 
this paper, as will be seen in the sequel. 

While the principal use of IPA has been in optimization, re¬ 
cently we considered an alternative application in performance 
regulation. Specifically, we addressed the problem of perfor¬ 
mance tracking in DEDS with time-varying characteristics. The 
feedback law we chose is comprised of an integrator with ad¬ 
justable gain, whose adaptation is based on the IPA derivative 
of the plant function with respect to the control variable. One 
of the key issues concerns the robustness of the regulation 
technique with respect to modeling uncertainties and errors in 
computing the IPA derivative (Wardi et al. (2015)). An applica¬ 
tion to throughput regulation in computer processors (Almoosa 
et al. (2012b)) highlights the importance of this issue since the 
controller has to run at short cycles, and therefore, if it is robust, 
it can be designed for simplicity and speed at the expense of 
accuracy. This approach was justified by simulation results in 
Almoosa et al. (2012b) as well as an analysis cited therein. 

Recently we considered an application of the IPA-based per¬ 
formance regulation technique to congestion management in 
traffic-light intersections, where the objective was to regulate 
the queue length in a given direction to a given reference (Wardi 
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Fig. 1. Control System 

and Seatzu (2014)). This paper extends the results therein in the 
following two ways: It considers vector tracking by a MIMO 
system while all previous results on IPA-based regulation con¬ 
cerned only SISO systems, and it considers the IPA derivative 
of a queue length at a given intersection by a control parameter 
at another intersection. In particular we compare, by simulation, 
the performance of a joint centralized controller for a two- 
intersection system with a decentralized control where each 
intersection is controlled by its own parameter. The results 
show similar performance for a range of parameters, which 
indicates the aforementioned robustness of IPA-based control 
and justifies the use of the simpler, decentralized scheme. 

Applications of IPA to road-congestion management have been 
addressed in Fu and Howell (2003); Panayiotou et al. (2005), 
and more recently in Geng and Cassandras (2012, 2013, 2015); 
Fleck and Cassandras (2014). Of course the traffic control 
problem has been amply researched for decades (see, e.g.. Fleck 
and Cassandras (2014) for a survey of techniques and results), 
and the IPA approach aims at on-line optimization using sample 
gradients in conjunction with stochastic approximation. The 
approach in this paper shares the principle of on-line sample 
gradients, but deviates from the above-mentioned approach 
in that it concerns performance regulation (tracking) and not 
optimization. In particular, it considers the control of queue 
lengths at traffic lights as a mean of congestion avoidance, and 
consequently the control laws that we propose are different. 

The rest of the paper is is structured as follows. Section 2 sum¬ 
marizes our regulation technique in general terms. Section 3 
sets the traffic control problem and analyzes the IPA derivatives. 
Section 4 provides simulation results, and Section 5 concludes 
the paper. All of the proofs are relegated to the appendix. 

2. REGULATION ALGORITHM: INTEGRAL CONTROL 
WITH ADAPTIVE GAIN 

Consider the n-dimensional discrete-time control system shown 
in Figure 1, where r £ R” is the setpoint input vector, k = 
1,2,..., denotes time, y k £ 1 " is the output vector, e k £ 1 " 
is the error signal vector, and u k £ R n is the input to the 
plant. Suppose first that the plant is a time-varying, memoryless 
nonlinearity of the form 

Vk = G k {u k ), ( 1 ) 

where G k : R™ —> R”, k = 1 , 2 ,..., is called the plant 
function. Given a reference input vector r, the purpose of the 
control system is to ensure that limfc_ s . 00 Vk = r. To this end we 
choose the controller to be a linear system defined as 

u k — u k —i T A k e k — i, (2) 

where A k £ R” xra , and the error signal is defined as 

efc = r — y k - (3) 


performance of the regulation scheme under variations in the 
plant. 

We define A k as 

= (^r K - l} ) • (4) 

Equations (1) - (4), computed cyclically in the order (4) —> 
(2) —y (1) —> (3), define the dynamics of the closed-loop 
system. 

The rationale behind the choice of A k in equation (4) can be 
seen in the fact that, if the plant is time-invariant and hence 
Gk{u) = G(u), this control law effectively implements the 
Newton-Raphson method for solving the equation G(u) = 
r. This observation was used in Almoosa et al. (2012a) for 
the single-variable control problem of regulating the dynamic 
power in computer processors. That reference also derived 
theoretical results concerning robustness of the tracking method 
to variations in plant modeling and errors in the computation of 
the gain. Moreover, general results concerning the robustness 
of the multi-variable Newton-Raphson method can be found 
in Lancaster (1966). We will rely on this robustness in the 
derivation of the IPA gradient in the sequel. 

To describe this control system in a temporal framework, when 
the plant is no more memoryless, let us divide the time axis 
{t > 0} into contiguous control cycles, Ti, T 2 ,..., in increas¬ 
ing order; I i starts at time t = 0, and for every k = 2, 3,..., re¬ 
starts at the same time 1 V_i ends. Suppose that the quantities 
Uk- i, y k - 1 , and e k -i have been computed or derived by the 
starting time of T k , and the following sequence of operations 
takes place during Yp. 

(i) A k has been computed via Equation (4) during the previ¬ 
ous control cycle, and it is available at the starting time of 

r fc ; 

(ii) Uk is computed by the controller at the start of A k via 
Equation (2), and the computation is assumed to be in¬ 
stantaneous; 

(iii) the plant acts on the input u k during IT, at the end of 
which it yields the corresponding output y k via Eq. (1); 
and 

(iv) e k is computed instantaneously by (3) at the end of Tfc. 

The most time-consuming computation can be expected to be 

that of Ak in Equation (4) since it involves the Jacobian matrix 

dG 

—i), implicitly assumed to be nonsingular. In the 
scenario discussed in this paper this is computed by the IPA 
derivative of the plant function, where we will make approx¬ 
imations designed to simplify the computations while relying 
on the aforementioned robustness. This will be discussed in 
detail in the next section. Due to the stochastic, dynamic, and 
time-varying nature of the system it cannot be expected to 
achieve a perfect performance tracking of a given reference. 
Instead, if the regulation algorithm converges faster than the 
rate of change of the system, we can expect the performance to 
chase the desired value in the sense that it approaches it rapidly 
between drastic changes. How well this works will be described 
in Section 4. 


Observe that, if A k is constant independent of k, then the above 
control law essentially is a multi-variable integrator (adder). 
Integral controllers often are associated with oscillation and 
narrow stability margins, therefore we chose a variable-gain 
integrator to extend the stability margins as well as to guarantee 


3. TRAFFIC REGULATION AT LIGHT INTERSECTIONS 

This section concerns traffic control on a road with two traffic- 
light intersections, but the analysis appears to be extendable to 
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Fig. 2. Two-light intersection 

a larger number of lights. At each intersection there is a control 
parameter associated with the traffic light as defined below, 
which can be used to regulate traffic-buildup (queue length) 
at the intersection. However, we also consider simultaneous 
regulation of the two queues. One of the objectives of this 
study is to compare the two control strategies. Whereas the joint 
control may be more accurate, the decentralized control of each 
intersection by its parameter is simpler. In the derivations of 
the IPA algorithms we make judgement calls about simplifying 
the computations whenever we feel it to be expedient, and we 
test the results by simulation. The rest of this section defines 
the problem and derives the IPA gradients that are used in the 
regulation scheme. 

3.1 Problem Definition 

Consider the two-intersection road system shown in Figure 2, 
where each intersection has a traffic light. Assume, for simplic¬ 
ity of argument, that each light cycle consists of red followed 
by green and there is no orange light. Let us focus on traffic in 
the direction of the horizontal arrows in the figure. Assume that 
traffic arrives at the first intersection according to a stochastic 
process {ai(t)}, and let S\(t) denote the instantaneous rate 
at which it enters the intersection (defined later). A fraction 
(f> = (j>{t) £ [ 0 , 1 ] of the process {^i(f)} proceeds to the 
second intersection while the rest follows other directions as 
indicated by the downward arrow. Also shown is the part of the 
cross traffic at the first intersection that is directed to the second 
intersection, represented by a stochastic process {ct 2 (t)}, which 
interferes and is multiplexed with the horizontal traffic crossing 
the first intersection. The superposition of the two flows com¬ 
prises the input-flow process {ct 2 (t)} to the second intersection 
from the left direction, and thus, 012 (f) = 4>(t)5i(t)+&2(t). For 
simplicity’s sake we assume no left turns at the intersections 
or, that left-bound traffic has its own turn signal and does not 
interfere with the traffic in the direction of the arrows that are 
shown. Thus, a green light in a given direction at an intersection 
corresponds to a red light in the perpendicular directions and 
vice versa. 

We consider a scenario in which a High-level (supervisory) 
controller has computed a traffic plan designed to manage 
congestion by balancing queue lengths, delays, and light-cycle 
times at the intersections. However, traffic bursts and other 
unpredictable events may cause traffic to deviate from its de¬ 
sirable behavior thereby leading to queue buildup. To mitigate 
these situations we regulate the intersections’ queue lengths 
by the duty ratios of the light cycles. We view this procedure 
as a mechanism for congestion avoidance which prevents the 
queue buildup at the second intersection from blocking traffic 
at the first intersection. Such acute congestion is assumed to be 
handled by a supervisory controller and is not discussed in this 
paper. 

The regulation technique that we describe is comprised of 
tracking the queue lengths at the two intersections to given 




reference values which are assigned, for example, by the super¬ 
visory controller. The underlying traffic model consists of the 
fluid-queue system comprised of two queues in tandem, repre¬ 
senting the respective queue buildup at the two intersections 
in the rightward direction shown in Figure 2. Let us denote 
the upstream queue by ql, and the downstream queue by q2. 
The inflow-rate process to ql and the outflow process from it 
are (ai(<)} and |<5i(f)}, respectively. A ^(f)-fraction of the 
outflow process proceeds to the second queue, where it is multi¬ 
plexed with the interfering process ( 0:2 (f)} to form the inflow¬ 
rate process there, {a. 2 {t)}, defined as 012 (f) = cj)(t) 6 i(t) + 
012 (f). 

Each of the queues has a constant light cycle of a given length, 
C\ and C 2 , comprised of red followed by green. The control 
variables of the system are the lengths of the red periods at 
the intersections, denoted by 0 \ and O 2 , respectively, and we 
define 9 := {9\ 1 # 2 ) T £ R 2 as the control vector. Naturally the 
length (duration) of the green periods are Ci — 8 i, i = 1,2. 
Note that some of the aforementioned rate processes depend on 
0 \ (like fii, 5i, and < 22 ), others depend on 62 (like fc), and a 1 
and 0:2 depend on neither 0\ nor Q-i- To simplify the notation 
we will denote their dependence on 9, as in /3i(f, 9), ct 2 (t, 9), 
etc. The queue-lengths (occupancy) will be denoted by x\ (f, 9) 
and X 2 (f, 9), respectively. 

During red periods at queue i, /3i(t,8) = 0. Upon a light 
switching from red to green, it is realistic to model the service 
rate as rising gradually rather than jumping to its highest rate. 
The definition of /3j(f, 9) reflects this in the following way: Let 
kCi be the starting time of the kth cycle at queue i, whose 
red period and green periods are the time-intervals \kCi , kCi + 
0i) and [kCi + 9i , kCi + Ci). Let bfir) be a positive-valued, 
monotone-increasing random function of r > 0. We define 
/3j(f, 9i ) = 6j(f — (kCi + 8i)) for every t in the green period 
[kCi + 9i, kCi + Cj ). Summarizing the definition of 9i) 
in both red and green lights, we have. 


0, if t £ [kCi, kCi + 8i) 

bi(t — kCi — 9i), if f £ [kCi + kCi + Ci). 


(5) 


Note that this definition of the service rate is quite general, and 
it includes the special case where (f, 9) holds a constant value 
during green periods. 


Based on the fluid models of the queues, the buffer lengths 
Xi(t, 9) are related to the inflow and service rate processes by 
the following one-sided differential equation 

dxi H m - / a *(*’ e ) - &(*> 0 )> if x i( f y e ) > 0 <f.\ 

W [t ' U) -\0, if Xi {t,8) = 0 , 


the outflow rate from the first queue, Si(t, 9), is defined by 




Pi(t,0), if xi(t,9) > 0 
ai (£), otherwise, 


(7) 


and the input process to the second queue is defined as 

a 2 (t, 9) = <j>(t)5i(t, 9) + a 2 {t). ( 8 ) 


The problem that we consider is to regulate the queue lengths 
by adjusting 9. Specifically, let us divide the time horizon into 
contiguous control cycles T*, = ["fk-i, 7 fc), k = 1 , 2 ,..., and 
let T fc := 7 k - 7 fc _i be the length of T k . Defining y iik := 
fr- J Fk Xi(t, 6)dt and y k := (y 1>k , y 2 ,fc) T , the objective is to 
regulate y k to a given reference vector. The regulation will be 









carried out via repeated applications of Equations (4) —> (2) —> 
(1) —>• (3) (with 9 replacing u) where the main challenge is 
to compute the gain A /,. defined in (4). This requires the IPA 
derivative of the plant function Gk (9) defined as 


G k {0) 


1 

T~k 



x\ (t, 9)dt 
X 2 (t, 9)dt 


L J 7fc-i 


(9) 


whose computation is specified in the next subsection. 


3.2 IPA Algorithm 


For the sake of simplicity in the forthcoming discussion we will 
omit the dependence of Gk{9), and other terms in (9), on k, and 
denote the generic fcth control cycle by the interval T = [0, T). 
By Eq. (4), it holds that 


A = 



( 10 ) 


where G(9) := (Gi((9), G2{0)) T is given by (9), and we next 
consider the computation of the IPA derivative Since both 
9 and G{9) are two-dimensional, we will be concerned with the 
four partial derivatives, ^ i , for i,j = 1,2. 


The case where i — j 


By Eq. (9) we have, for i = 1,2, that 

i rT 


GM) = 


•^i (f; 9i)dt, 


0 


and since Xi(t, 9i) is continuous in i, 


8G 


dxi 


p,a~rr aa ®i)dt. 


89, 


T 


89i 


( 11 ) 


( 12 ) 


Reference Wardi and Seatzu (2014), considering only a single¬ 
queue system, derived the following result for the term Bfp- ■ 
For t lying in the interior of an empty period in queue i, 
^ L (t, 9j) = 0. On the other hand, for t lying in the interior 
of a busy period in queue i, let ut be the starting time of the 
busy period containing t. Recall that kCi is the starting time of 
the kth red period at queue i, and let kCi, k = £,..., m, be 
those points lying in the interval [ut, t). Then 


wA A) = 


,9i) + Pi(t,9i) - Pi(u t (9i) + ,9i). (13) 


k=i 


We point out that the rate-terms in (13) can be measured in real 
time by detecting the speed of passing vehicles. For details of 
these derivations please see Wardi and Seatzu (2014). 


Eq. (13) is especially simple when f3i(t,9) is equal to a given 
constant Bi > 0 during green-light periods at queue i. In that 
case it is readily seen that the first sum-term in the RHS of 
Eq. (13) is equal to (to— £+l)Bi, 9B) G {0, Bi] depending 

on whether t is in a red period or a green period, respectively; 
and similarly Bi{ u t(9i) + ,9i) G {0, Bi] depending on whether t 
is in a red period or a green period, respectively. In this case the 
computation of BjA- is a matter of a simple counting process. 


first queue, and therefore xi(t, 9) is a function of 0\ but not of 
9 2 . Consequently, = 0. 

We next consider the partial derivative term 9). We will 

derive for it an event-based algorithm, where the events in ques¬ 
tion are light-switchings, jumps (instantaneous discontinuities) 
in traffic rates, and the beginning and end of busy periods at the 
queues. We say that two events are independent if neither event 
causes the other to occur at the same time. The following as¬ 
sumption is quite common in the literature on IPA of stochastic 
hybrid systems (e.g., Cassandras et al. (2002, 2010)). 

Assumption 1. For a given control variable 9 G R, w.p.l, no 
two independent events occur at the same time. 

We also implicitly assume that all of the derivative terms 
mentioned in the sequel exist w.p.l, and point out general and 
verifyable conditions guaranteeing this assumption (Cassandras 
et al. (2002, 2010)). 

Before deriving the term ^-(t,9) we mention four types of 
approximations that can be practical in implementations. These 
approximations will be used for computing the IPA derivatives 
but not for analysis of the traffic flows themselves which 
comprise the state of the “real” system. Therefore we expect the 
regulation scheme to work for a range of errors because of its 
aforementioned robustness. Some of these approximations are 
tested via simulation in Section 4 while others are the subject 
of current research. In the first approximation we assume that 
instantaneous traffic rates can be measured as, for instance, by 
speed detectors that are commonly used in traffic monitoring. 
Second, the fraction-term (f>{t) is a mathematical construct 
that is well defined only for the fluid-flow model but not 
for the “real”, discrete system, and hence we replace it by 
a term <f> G [0,1] that can be computed, for example, by 
averaging traffic flows taken from real-time measurements. The 
derivative term ^(£) is neglected in the computation of the 
IPA derivative. Third, as mentioned earlier, we will test the 
decentralized version of the controller on traffic obtained from 
the correlated two-queue system. Fourth, we neglect in the 
forthcoming analysis the effects of delays in the control loop, 
which will be addressed in the future. 

As for the term J(f, 9), consider separately the cases where t 
lies in the interior of an empty period vs. a busy period in q2. If 
t lies in the interior of an empty period in q2 then obviously 

Sr<‘- 9 >=»- " 4 » 

Consider next the case where t lies in a busy period in q2, and 
denote by p = p{9) the starting time of this busy period. Then 

X2(t) = f (a 2 (r,0) - j8 2 (T,0))dr, (15) 

Jp 

and therefore 

^M) = -^-(/ (ol 2 (t,9) -/3 2 (r,0))dr). (16) 

Suppose first that p lies in a ql-green period including the start 
of such a period, and that t lies in a ql-green period as well. 1 
Let [ 77 *, £i), i = 1..... A:, denote the ql-red periods contained 
in the interval [p, t) in increasing order, and define £0 := p and 


The case where i ^ j It was mentioned earlier that we do not 1 We use the terms “ql-green period”, “ql-busy period”, etc. to mean a green 

consider the case where the second queue blocks traffic at the period in ql, a busy period in q2, etc. 





r]k+i = t, so that the intervals 77 ,;), i = 1 ,..., k + 1 , are 
ql-green periods contained in the interval [p, t]. 

Lemma 1. The following equation is in force. 


dx 2 

Wi 


M) 


= -( a 2 (p + ,0 ) - /3 2 (p + ,9 )) 


dp 

d(h 


+ X]( a2 ^i > 0 ) -«2(C> 0 )) 


fc+1 


+ [ + (“ 2 (t, 9) - (3 2 (T,8))dT. 

i= 1 1 


(17) 


Moreover, the first term in the Right-Hand Side (RHS) of (17) 
is equal to 0 except in the following situation: the start of the 
q 2 -busy period at time p is triggered by a jump up in /?i(s, 9) 
at the same time, s = p. In this case = 1, and the first term 
in the RHS of (17) is equal to — {a 2 (p + , 9) — (3 2 (p + , 9 )). 


The proof can be found in the appendix. □ 

Remark 1. If a point r £ [0,T] lies in the interior of a ql- 
red period then a 2 (r, 9) = a 2 (t, 9 2 ), which is independent of 
9\. Therefore, if q lies in the interior of a ql-red period then 

(a 2 (r,9) — /3 2 (t, 9))dr = 0. In this case Equation 
(17) remains the same except that the sum in the last term of 
its RHS starts at * = 2 instead of i = 1. Likewise, if t lies in 
the interior of a ql-red period then the last term of that sum is 
i = k and not i = k + 1. 


J^ + (a 2 (t,9) - p 2 (t,9))dt = 



+C’(/3i(t 6 + i J - PiiVi )) • (20) 

(ii). If £i_i is contained in the interior of a ql-empty period 
then the first additive term in the RHS os (20) is zero, (iii) If r]i 
lies in a ql-empty period then the last additive term in the RHS 
of (20) is zero. 

The proof can be found in the appendix. □ 

Remark 2. In (20), the term is assumed to be com¬ 
putable via (13) as applied to ql. It holds that = 1 for 

i > 1 (since £i_i = kC\ + 9\), and for i = 1 

(since £ 0 = P )■ 

The analysis leading to Lemma 1, Lemma 2, and the remarks 
that follow them imply that Algorithm 1, below, computes the 
IPA derivative 9) for / lying in a q2-busy period. By (9), 
this will complete the computation of ^§f{9). The algorithm’s 
description focuses on a single q2-busy period, where we use 
the notation defined for the analysis in the earlier paragraphs. 
Thus, let p be the starting time of the busy period and, in 
increasing order, let ip and respectively, i = 1, 2,..., be 
the starting times of ql-red periods and green periods during 
the ql-busy period begun at time p. If p lies in a ql-red period 
then we set p\ = p, while if p lies in a ql-green period then we 
define £o = p and in this case ip > p. In either case, £1 > P- 


The first two terms in the RHS of (17) involve flow rates which 
are assumed to be measurable. It remains to assess the third 
term in (17), which we next are set to do. 

Consider the term 

^- J^ (& 2 {t, 8) - /3 2 {T,9))dT, (18) 

and recall that the interval r/i) is a green period in ql. Let 
us partition this interval into busy periods and empty periods 
in ql. Thus, define [Tbp,T e ,i) as the first ql-busy period in 
the interval 77 *), [r e !, 2 ) is the following empty period, 

followed by the next busy period [Tb, 2 ,T et2 ), etc, and let r 6im 
denote the end-point of the last ql-busy period in the interval 
[£i_i, r)i\. Observe that if lies in a ql-busy period (or is 
the starting time of such a period) then Tf,j = 1 , and if 1 

lies in a ql-empty period then 77 ,. 1 > £i_i. Likewise, if r/i lies 
in a ql-busy period then r e m = r/, , and if t lies in a ql-empty 
period then r e m < Pi- 

Lemma 2. (I). If the interval [£i-i, Pi\ is contained in a single 
ql-busy period then 

J^ + (a 2 {T,9) -/3 2 (T,9))dT 

= <KPii€t-i ,0)-/M%,0)). 09) 

(II). Suppose that the interval [£*_!, 77 j] is not contained in a 
single ql-busy period, (i). If both £j_i and 77 i are included in 
two different ql-busy periods then 


The algorithm computes, recursively, quantities D t at the times 
Pi, and quantities E, at times £j. Lurthermore, for every t £ 
[Pii CO (ql-red period) it sets 9) = Di, while for every 

t e [£i,Pi+ 1 ) (ql-green period) it computes a function <^(f, 9) 
(defined below) and then sets = Ei + gi(t). The algo¬ 

rithm has the following form. 

Algorithm 1: 

• At time p: 

If p lies in a ql-red period: Set ip = p, set D\ = 0. Lor 
every t G [pi,£i), set f§*(f,0) = 0. 

On the other hand, if p lies in a ql-green period, set 
Co = P, and set E f> as follows: If the start of the busy 
period is triggered by a jump /3i(-), and p lies in a ql- 
green and busy period including the start of such a period, 
set 

E 0 = -((j)P 1 {p,6)-P 2 {p,0))\ ( 21 ) 

otherwise, set Eq = 0. Moreover, for every t G [Co,7i]> 
set 

d 

-^-(t,0) = E o + go(t), ( 22 ) 

where go(t) is defined below. 

• At time p^ i = 2,...: 

Set 

Di = Ei_i + <7i_i(?7i), (23) 

where the function g^_i (t) is specified later. Moreover, for 
every t G [??»,&]. set 


( 24 ) 





Note that the situation where i = 1 corresponds to the 
case where p lies in a ql-red period, which was discussed 
earlier. 


• At time i = 1,2,...: 

Set 

E i = D i + a 2 (C, 9) - a 2 (g , 9 ). (25) 

Moreover, for every t £ [£*, r]i+i\, define 
dxo 

— (t,0) = Ei + gi (t ), (26) 

where the function gfit) is defined as follows. 

• Definition of gfit) for t £ [£,;, r)i\, i = 0, 1, .. .: 

If t in the interior of a ql-empty period, set g,(f) = 0. 

On the other hand, if t lies in a ql-busy period, we set 
gi(t) = <7i,i(i) + g%, 2 {t), where the functions i and 
g, 2 are defined as follows. g i: \ (f) depends on whether 
lies in the interior of a ql-empty period or in a ql- 
busy period. If lies in the interior of a ql-empty period 
then = 0. On the other hand, if lies in a ql- 

busy period (including the start of such a period), then we 
define 


= - (a^e) p^t,e ))-^j 

(27) 

We point out that for every i = 1,2,..., the last term in 
(28) is = 1. In the special case where fo = P, we 


have that = 1 as long as p is a point in a green, busy 
period at ql, and the start of the q2-busy period begun at 
time p is due to a jump up in at time jr, in all other cases 


9£o _ Q 
90 ! U - 


Consider next the function g ,/2 (f) . If t lies in the interior 
of a ql-empty period then gi, 2 (t) = 0. Likewise, if lies 
in a ql-busy period and t lies in the same busy period, 
then gi, 2 (t) = 0. For the remaining case t lies in a ql-busy 
period to which does not belong. In this case, define Tj.(, 
the starting time of the ql-busy period containing t. Then, 
gi, 2 (t) is defined as 


9i,2{t) = <A(/?i(Lg,, 0 ) (28) 


□ 


Remark 3. The algorithm looks quite complicated due to the 
need to track several types of events, but it is quite simple 
to code. Considerable simplification is likely to result from 
the likelihood that a single ql-green period is either contained 
in a single ql-busy period, or divided into a ql-busy period 
followed by a red period. It would be even simpler if the service 
rates at the queues have the form 


Pi(t) 


0, if t lies in a ql — red period 

Pi,max 7 if t lies in a ql — green period. 


(29) 


for given constants f3i, max > 0, since in this case fifit, 6) would 
be determined by whether t lies in a red or green period. Even 
if this is not the case for “real” traffic, such a reasoning can be 
used in the IPA algorithm. 


4. SIMULATION EXAMPLES 


This section presents simulation examples for testing the effec¬ 
tiveness of the proposed regulation technique. The traffic-light 
cycles are is C\ = C 2 = 1, and each control cycle consists 
of 20 light cycles. The process {ai(i)} consists of an off/on 
model where, in the off stage Qi(f) = 0, while for each on 
stage oq (f) has a single drawn value uniformly distributed in 



Fig. 3. Evolution of Gi k (thick line) and G 2 k (thin line) for 
k = 1 ...,50 



Fig. 4. Evolution of Gi k (thick line) and G 2 k (thin line) for 
k = 10 ...,50 

an interval [(1 — C)di, (1 + C)®i]; we chose its mean to be 
di = 4.1, and set ( = 0.3. The durations of off periods 
and on periods are drawn from the uniform distributions on 
the intervals [0, 0.02] and [0, 0.063], respectively. The process 
{ 52 (f)} is generated (drawn) in a similar way except that its 
mean value is a 2 = dq/10 = 0.41. This is motivated by the 
fact that the external arrival in the second queue is considered 
as noise that is not regulated by the traffic light. The main 
input flow in the second queue is assumed to come from the 
first queue, where we took </> = 0.9. The service-rate processes 
{Pi(t,9 )}, satisfy Equation (29) with Pi,max = 5.0, i = 1,2. 
The set-point reference vector is r = [0.1, 0.1] T , and the initial 
control variables were set to 9\ = 0 2 = 0.8. 

Fig. 3 depicts the graphs of the obtained outputs G\,k(9) and 
G 2 ,k{0) as functions of the counter k = 1,..., 50, and we 
observe convergence in about 5 iterations. Fig. 4 provides the 
same information for /.: = 10,..., 50 in order to highlight a 
variability of the output about the target values of 0.1, which 
is due to the randomness in the system. However, the respec¬ 
tive means over the last 41 iterations, namely the quantities 
JT Gi,k(9k), are 0.1001 and 0.0986fori = 1 andi = 2, 

respectively. 

Figure 5 shows plots of the control variables 0 t j-, k = 1..., 50, 
and we discern convergence to their respective values around 
6\ ~ 0.3113 and 0 2 ~ 0.4129. It is not surprising that the 
asymptotic value of 0 2 is larger than that of 0 \. The reason is 
that the input processes to the two queues have the same mean 
rate, but {a 2 (t, d)} has less variance than {ai (f, 0)} due to the 
action of the first queue. Therefore, to obtain the same mean 
queue lengths the second queue would have a larger traffic 
intensity and hence smaller mean service rate, meaning that 
e 2 > 0i. 

A similar behavior, not shown here, was obtained with different 
values of the initial control variables as well as different values 
of the traffic parameters ( and cj>. 









mean\Gi(10 : 50) — ri| 

mean|G2(10 : 50) — r 2 \ 

maxGi(l : 50) 

max G 2 (l : 50) 

£ = 0.05, centralized 

0.0843 ■ 10“ 3 

0.2575 • 10-3 

0.3507 

4.7475 

decentralized 

0.0855 ■ 10“ 3 

0.0275 • 10-3 

0.3481 

4.2749 

£ = 0.10, centralized. 

0.0737 ■ to” 3 

0.1726 ■ 10-3 

0.7258 

1.1770 

decentralized 

0.0637 ■ 10 —3 

0.0415 • 10“ 3 

0.6633 

1.9848 

£ = 0.15, centralized. 

0.1505 ■ 10“3 

0.0753 • 10-3 

2.6368 

1.9309 

decentralized 

0.2167 ■ 10“3 

0.0267 • 10-3 

2.5817 

3.6931 

£ = 0.20, centralized. 

0.1917 ■ 10-3 

0.4577 • 10-3 

4.9831 

3.8481 

decentralized 

0.2094 ■ 10-3 

0.1458 ■ 10“3 

5.1314 

7.7879 

£ = 0.25, centralized 

0.2642 ■ 10-3 

0.6798 ■ 10-3 

7.2245 

2.2578 

decentralized 

0.2731 ■ 10-3 

7.4621 

7.4038 

18.3908 

£ = 0.30, centralized 

0.4617 ■ 10-3 

0.2643 ■ 10-3 

9.9984 

3.0413 

decentralized 

0.4164 ■ IQ -3 

8.4869 

10.0046 

20.9073 


Table 1. Comparison between centralized and decentralized regulation scheme. 



Fig. 5. Evolution of 0\.k (thick line) and 0 2t k (thin line) for 
k = 1 ..., 50 

We compared the joint, two-queue regulation scheme described 
in the last two sections with the decentralized control in which 
each queue computes its own gain (via (4)) without considering 
the effects of ql on q2. The main difference is that in the second 
case the Jacobian ^ is diagonal, and hence its computation 
is made much simpler: compare (13) to Algorithm 1. The 
comparison was made for various values of the input variance 
related to different choices of the traffic parameter The 
results, shown in Table 1, comprise averages of 10 independent 
runs for each indicated value of £, where in each run the first 
central moments of the output (columns 2 and 3) are computed 
over k = 10 — 50 in order to avoid the effects of the early 
transients, while the maximum deviations (columns 4 and 5) are 
taken over fc = 1 — 50. It is evident that for lower variance £ < 
0, 2 the performances of the decentralized control is comparable 
to that of the centralized control. However, as the variance 
grows the errors associated with neglecting the derivative 
render the decentralized control unstable while the centralized 
controller works well. 

5. CONCLUSIONS 

The main contribution of this paper is in a flow control tech¬ 
nique in traffic-light intersections which aims at regulating 
queue lengths to given reference setpoints. The technique is 
based on multivariable integrators with adaptive gains com¬ 
puted using IPA. Numerical simulations are presented to cor¬ 
roborate the effectiveness of the proposed approach. A simpler, 
decentralized approach based on approximations designed to 
reduce the computing efforts is also considered, and its perfor¬ 
mance is shown (via simulations) to be similar to that of the 
multivariable integrators for a range of parameters. 


6. APPENDIX 

This section provides proofs to Lemma 1 and Lemma 2. 

Proof of Lemma 1. By (16), Leibnitz rule, and the fact that 
r)i -(_i = t is independent of 0\, 

(t , 0) = - (a 2 (p + , 0) - (P + , 9)) 

k 

+(° 2 fa * 7 >- fa far i e ) 

i— 1 

— («2(??+, 6») - 0i))) 

k 

+ £( a 2(C>0)-&(£->*) 

i= 1 

k -\-1 — 

+ H /+ (“2 (T,O)-P2(T,0))dT 

i—1 1 J ti-l 

k f) r$i 

+ (a 2 (r,0)-/3 2 (t,0))<2t. (30) 

i=i dQl V 

Light-switchings are events and hence, and by Assumption 1, 
the functions <5 2 ( t ) and /3 2 ( t , 0) are continuous at t = rji 
and t = £i, i = 1,... ,k. Next, rji = fcCj and = kC\ + 
0i for some k = 1 ,..., hence = 0, and = 1 , 
rendering the first sum-term in the RHS of (30) to 0, and the 
last multiplicative term in the following sum-term to = 
1. Lurthermore, during ql-red periods a 2 (r, 0) — /3 2 (t,8) = 
a 2 (t) — j3 2 (t, 0 2 ) which is independent of 0i, and hence the 
last additive term in (30) is zero. In light of all of this. Equation 
(30) gives (17). 

The last assertion of the lemma is obvious. □ 

Proof of Lemma 2. (I). Suppose that the interval t?,] is 
contained in a ql-busy period. By definition, it is also contained 
in a ql-green period. Therefore, for every r in this interval, 
a 2 (r, 0) = (r, 0) + a 2 (r, 0). Moreover, by definition and 

Assumption 1, the functions <5 2 (t) and /? 2 (r, 0 2 ) and their jump 
times are independent of 0i. Consequently 
















^ J^ + (c*2(t,9) -p 2 (T,0))d,T = Pi(r,9)dr. 

But for every r G [&_i ,rji] /3i(r, 0) = &i(r - (fcCi +9i)) and 
hence ^r(r, $i) = — 6i), which implies (19). 

(II). Consider case (i) where Q and r; 7 lie in different ql- 
busy periods. The times Tb,j and r ej are event-epoch in ql 
and hence, and by Assumption 1, the functions P 2 {t, 0) and 
<22 (t) are continuous at these points. Next, rji is a switching 
time from green to red in ql, hence rji = kCi for some 
k = 1,..., and therefore = 0. Furthermore, recall that the 
interval [£i_i• pf) is contained in a ql-green period. Then for 
every r G j = 1 ,... ,m, a 2 {r,9) = </>/3i (t,9) + 

d 2 (r), and for every r G (r e j, n,j+i), j = 1, ...,m - 1, 
a 2 (r,9) = </>ai(r) + < 22 (t); this is independent of 9\ as is 
/3 2 (t, 9) and hence (a 2 (r, 9) - /3 2 (t, 6)) =0. Applying all 
of this with Leibnitz rule we obtain 


99i y e +_ i 


(a 2 (r, 0) - /3 2 (T,0))di 

dn,j 




J =2 
m —1 


+ H (/ J l( T ed> 6 ') “ a l( T ej)) 


dr, 


e,j 


f=l 


96*i 


+ <j>Pl{T,9)dT 

j=l •* T b,j 


(31) 


The first sum-term in the RHS of (31) is zero for the following 
reason: The point Tb,j is the starting time of a ql-busy period. If 
it is triggered by a jump up in a.\ (r) at r = t& j then 5^. = 0 
since a.\ (r) is independent of 9\, it cannot be triggered by a 
jump down in /3i(r, 6) since this implies the end of a ql-green 
period, but we assume that the interval [Q_i, hi) is contained 
in a ql-green period; and if it is due to a continuous rise of 
< 2 i(t) — 0i(t,9) from negative to positive then atifoj) — 
Pi (n,j , 9) = 0. 


Next, for every j = 2,..., m — 1, the interval ( i~b,j,T et j ) com¬ 
prises a ql-busy period, and hence j^’° (r)— / (r, 0)) dr = 

0. Taking derivatives with respect to 6 * 1 , we obtain. 


9 r T *>i Ot 

~W 1 J+ ^ 1 ( r > 6 ') dr +( ai ( T e,f)-/ 3 l ( T e,f : 6 '))^ 1 = 0 - 

T b,j 

(32) 

The case j = 1 is different in that (<21 (r)— /3i(r, 9)) dr = 
—Xi(£i-i,9), and similarly to the derivation of (32), we obtain. 


(< 2 1 (r ejl ) -Pi( t Bi1 ,9) - (ai(Cjti) ~ 

By Equation (31) with the aid of (32) and (33), Equation (20) 
follows. 
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Finally, parts (II.ii) and (Il.iii) of the lemma are obvious in 
light of the proof of part (II.i), since during a ql-empty period 
X\ (t, 9) = 0 and neither a 2 (r, 9) nor /3 2 (r, 9) depend on 9±. □ 







